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A  SET  OF  DEBUGGING  AND  MONITORING  FACILITIES  TO  IMPROVE  THE 
DIAGNOSTIC  CAPABILITIES  OF  A  COMPILER 

Elizabeth  Fong 

Increasing  concern  with  the  quality  of  computer  soft- 
ware today  makes  it  important  to  evaluate  critically  the 
debugging  facilities  available  in  high-level  languages. 
This  paper  presents  a  collection  of  program  debugging  and 
monitoring  facilities  to  improve  the  diagnostic  capabilities 
of  a  compiler.  A  distinction  is  made  between  debugging  and 
monitoring  facilities  performed  at  compile  time,  at  link/ 
load  time  and  at  execution  time.  These  facilities  are 
described  in  terms  of  this  breakdown  with  a  conscious  at- 
tempt to  move  the  detection  of  errors  from  execution  time 
to  compile  or  link/ load  time,  and  to  collect  information 
when  the  information  is  available  during  the  compilation 
process . 

Key  words:  Compiler;  debugging;  error  diagnostic;  high- 
level  programming  languages;  monitoring; procedural-oriented 
languages . 

1.  INTRODUCTION 

Software  production  requires  large  amounts  of  both  human  and  machine 
resources.  The  recent  trend  has  been  towards  a  more  systematic  devel- 
opment of  software  and  in  particular  towards  producing  "quality"  soft- 
ware that  is  effective  and  maintainable.  During  the  program  development 
stage,  most  of  the  programmer's  effort  is  spent  in  debugging.  Program 
debugging  is  still  a  very  uncertain  process.  Professionals  do  not  agree 
on  how  to  get  the  bugs  out  of  a  program  systematically,  or  how  to  con- 
struct the  most  reliable  program.  Tools,  both  automated  and  manual, 
exist  but  surprisingly,  much  of  the  work  in  this  field  has  been  des- 
cribed only  in  unpublished  reports  or  passed  on  through  oral  tradition. 
Interest  in  debugging  techniques  has  .-been  aroused  recently,  and  a  col- 
lection of  papers  devoted  to  the  topics  of  debugging -jand  maintenance  of 
large  software  systems  can  be  found  in  Reference  [1]. 

The  programming  activity  is  rooted  in  the  traditional  environment, 
that  is,  writing  the  program  in  a  high-level  programming  language,  such 
as  FORTRAN,  ALGOL,  COBOL,  or  PL/1  with  the  usual  compile- link/ load-go 
situation.  To  aid  the  programmers  in  the  stage  of  program  development, 
it  is  advantageous  to  build  debugging  aids  and  monitoring  features  into 
a  high-level  programming  language  system. 

This  paper  presents  a  collection  of  program  debugging  and  monitoring 
facilities  to  improve  the  diagnostic  capabilities  of  a  compiler.  A  dis- 
tinction is  made  between  debugging  and  monitoring  facilities  performed 


1Figures  in  brackets  indicate  the  literature  reference  at  the  end  of 
this  paper. 


at  compile  time,  at  link/ load  time  and  at  execution  time.  These  facil- 
ities are  described  in  terms  of  this  breakdown  with  a  conscious  attempt: 

to  move,  as  much  as  possible,  the  detection  of  errors  from  execu- 
tion time  to  compile  or  link/ load  time  in  order  to  avoid  overhead 
in  an  executing  program,  and 

to  collect  information  when  the  information  is  available  during 
the  compilation  process.  At  the  user's  option,  this  information 
could  be  output  at  a  later  time. 

2.   SCOPE  AND  SOURCES 

Many  programmers  think  of  debugging  aids  as  "dumps"  (the  display  of 
the  contents  of  storage  cells)  and  "traces"  (the  display  of  the  control 
flow  during  execution).  However,  these  basic  tools  have  become  consider- 
ably more  sophisticated  over  the  years.   In  particular,  very  powerful 
debugging  faciliti.es  have  been  built  for  the  on-line  interactive  envi- 
ronment [2,3]  giving  the  user  a  high-level  language  with  which  to  control 
the  setting  of  breakpoints  and  to  allow  interrogation  of  individual 
storage  cells. 

The  universities  in  teaching  programming  to  students,  have  found 
that  they  need  an  extensive  set  of  debugging  facilities.   "Forgiving" 
compilers  have  emerged.  These  include  not  only  error  detection  but 
error  correction  features  as  well.  Examples  include  Cornell  Computing 
Language  (CORC)  [4],  Waterloo  FORTRAN  (WATFOR  and  WATFIV)   [5],  and 
Purdue  Fast  FORTRAN  Translator  (PUFFT)  [6]. 

Program  monitors  are  another  programming  aid.  These  can  be  viewed 
as  a  logical  extension  of  the  debugging  aids  described  above.  A  pro- 
gram monitor  is  a  program  built  to  measure  the  behavior  of  another  pro- 
gram. In  general,  the  program  monitors  that  are  available  commercially 
operate  in  two  phases:  data  gathering  and  data  presentation. 

The  idea  of  combining  debugging  and  monitoring  facilities  as  an 
augmentation  to  high-level  languages  has  been  attempted  in  the  EXDAMS 
Project  [7].  Both  static  and  dynamic  displays  may  be  exhibited  at  the 
request  of  the  programmer. 

Dijxstra's  notion  of  structured  programming  [8],  and  Wirth's  concept 
of  program  development  as  the  process  of  successive  refinement  [9]  are 
attempts  to  move  programming  from  a  private  art  to  a  new  level  of  pre- 
cision. The  essential  ingredient  in  achieving  a  correct  final  program 
quickly  seems  to  lie  in  the  area  called  "good"  programming  practices. 
Examples  of  "good"  programming  practice  include  the  exhibition  of 

structured  readable  source  statement  as  output,  planting  special  checks 
and  messages  for  the  use  of  undeclared  variables,  and  elimination  of 
wild  GOTO  transfers,  etc.  Some  of  these  ingredients  could  be  implemented 
on  a  compiler. 


3.   DESCRIPTIONS  OF  DEBUGGING  AND  MONITORING  FACILITIES 

In  a  compiler  environment,  the  program  text  is  coded,-  keypunched  and 
fed  into  a  compiler  which  produces  object  code.  The  object  code  and  a 
system  library  are  link-edited  and  loaded  by  a  link/loader  which  produces 
absolute  code  ready  to  be  executed.   (see  Figure  1).  Three  distinct 
times  can  be  identified  at  which  debugging  and  monitoring  can  occur: 
compile  time,  link/ load  time,  and  execution.  Each  individual  type  of 
check  is  described  in  terms  of  this  breakdown. 
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Debugging  and  monitoring  features  described  below  are  categorized 
according  to  these  stages.   Some  features  may  occur  either  at  compile 
time  or  link- load  time  depending  on  the  implementation.  This  three-part 
breakdown  hypothesizes  that  the  earlier  an  error  is  caught,  the  easier 
it  is  to  deal  with.  Therefore,  any  tool  or  technique  that  moves  detec- 
tion of  certain  errors  from  execution  time  to  compile  time  is  valuable. 
This  breakdown  does  not  apply  to  the  interpretive  environment. 

3.1  Compile  Time  Checks 

The  types  of  checks  performed  by  the  compiler  are  usually  referred  to 
as  static  checks  because  the  problems  they  address  are  invariant  with 
execution  time  and  independent  of  the  values  of  the  input  data.  The  fol- 
lowing are  descriptions  of  some  program  debugging  and  monitoring  facil- 
ities that  could  be  performed  at  compile  time. 

3.1.1  Syntax  Checking.   One  of  the  first  tasks  done  by  a  compiler  is 
to  scan  the  source  program  statements.  Every  high-level  lan- 
guage has  its  own  established  set  of  syntax  rules.  Error 
situations  occur  when  the  compiler  cannot  translate  a  source 
statement,  or  the  process  of  syntax-directed  parsing  is  blocked. 


Examples  of  syntax  errors  include  misspelling  of  reserve  words, 
punctuation  errors  such  as  unmatched  parentheses,  missing 
operators ,  illegal  sequences  of  operators  in  expressions ,  or 
missing  statement  labels.  In  a  class  of  university-developed 
compilers  (e.g.,  CORC,  PUFFT,  WATFOR)  with  "forgiving" 
features  built  in,  extensive  syntax  checking  is  done.  These 
compilers  also  do  some  error- correction  during  source  language 
scanning.   For  example,  CORC  attempts  to  define  various  cate- 
gories of  misspelling  and  always  tries  to  repair  the  error  and 
continue  to  compile  executable  code.  All  compilers  have  syntax 
checking  facilities,  however  the  extent  of  checking  varies. 

3.1.2  Static  Concordance  of  Variables  and  Labels.  At  compile  time , 
all  the  variables  and  labels  are  entered  into  a  symbol  table 
or  an  intermediate  file.  This  information  could  be  made 
available  during  compile  time  to  produce  one  of  the  most  bene- 
ficial aids  to  debugging.  This  facility  is  sometimes  called  a 
cross  reference  listing.  The  usual  format  consists  of  the 
symbolic  name  followed  by  the  concordance  of  the  line  numbers 
where  the  variable  is  being  defined  and  the  line  numbers  where 
this  variable  is  being  referenced.  Some  compilers  also  print 
out  data  type  and  relocation  information.  An  example  appeared 
in  Figure  2.  In  this  case,  the  statement  label  concordance 
appeared  as  a  separate  listing.  The  advantages  of  such  a  fac- 
ility would  be  to  detect  a  variable  or  a  label  being  defined 
but  never  referenced  or  vise  versa.  For  a  block-oriented 
language  such  as  ALGOL  60  or  PL/1,  the  variables  and  labels 
may  be  declared  as  local  within  a  block.  Exhibiting  a  static 
concordance  of  variables  and  labels  plus  their  scope  of 
definition  would  aid  debugging.  Most  of  the  compilers  provide 
this  facility.  There  are  also  programming  support  packages 
available  which  can  generate  such  a  concordance. 

3>1-3  language  Flagging  Features.  Due  to  historical  development, 
most  programming  languages  (e.g.,  FORTRAN)  have  undergone 
modifications  and  upgrading  (e.g.,  FORTRAN  II,  Basic  FORTRAN, 
FORTRAN  IV,  and  extended  FORTRAN).  Moreover,  each  implemen- 
tation of  the  language  may  contain  differences.  Flagging  can 
be  done  during  the  source  language  scanning  phase  to  detect 
language  syntactic  features  which  deviate  from  some  "standard" 
definition  such  as  ASA  FORTRAN  [10],  or  additional  features 
from  the  previous  versions.  A  message  could  be  printed  out 
by  the  compiler  as  a  warning  to  the  user  that  the  particular 
feature  is  unique  to  this  version  of  the  compiler.  Very  few 
compilers  have  this  flagging  facility.  An  idea  closely  re- 
lated to  this  can  be  found  in  a  COBOL  compiler  which  prints 
the  message:  "REDEF  Name  feature  not  the  same  as  previously 
defined  version".   (See  Figure  3).  It  is  possible  to  deter- 
mine the  language  feature  differences  by  compiling  ±he  prev- 
iously compiled  program  with  the  new  compiler,  however,  the 


point  here  is  to  detect  deviations  specifically  against  a 
"standard"  language  definition,  or  to  be  able  to  identify 
a  subset  of  the  language  that  existed  earlier.  To  make  a 
full  semantic  check  of  a  program  is  not  possible. 

3.1.4  Logical  Segmentation  of  Programs.  A  large  program  may  be 
considered  as  consisting  of  small  logical  segments.  This 
permits  a  modular  organization.  Modularity  aids  debugging 
because  it  breaks  up  the  program  into  manageable  sized  pieces. 
In  ALGOL,  there  are  BEGIN-END  statements  which  create  a  logi- 
cal segment.   In  COBOL,  a  paragraph  or  a  section  also  is  con- 
sidered to  be  a  logical  segment.  A  formatting  facility  could 
be  built  into  the  compiler  which  would  cause  more  readable 
source  program  statements  to  be  printed  out.  Examples  are.: 
alignment  of  BEGINs  with  corresponding  ENDs  with  nested  inner 
blocks  properly  indented;  hierarchical  data  declaration  in 
COBOL  or  PL/1  are  properly  indented  to  reflect  structures 
even  when  the  input  is  not  punched  in  the  proper  column;  or 
reordering  of  source  statements  in  order  to  group  declaration 
statements,  executable  statements,  and  format  statements. 
Figure  4  shows  an  example  of  an  ALGOL  program  printout  with 
block  number  and  level  number  generated  by  the  compiler  to 
reflect  logical  segments.  This  facility,  although  not  a  de- 
bugging feature  in  the  sense  that  it  helps  to  uncover  errors, 
could  greatly  add  to  the  readability  of  a  program.  Producing 
readable  documentation  also  helps  another  person  in  maintain- 
ing the  program. 

3.1.5  Static  Control  Structure  Concordance.  The  normal  flow  of  con- 
trol xn  a  program  is  sequential.  However,  control  may  be 
transferred  elsewhere  in  the  program  by  a  transfer  of  control 
command,  e.g.,  GOTO,  RETURN.  A  topological  structure  diagram 
showine  everv  transfer  of  control  at  the  request  of  the  user 
would  aid  debugging.  Figure  5  contains  such  a  static  control 
structure  printout,  showing  that  a  transfer  of  control  is 
occurring  at  line  number  n  to  line  number  m.  Some  compilers 
do  not  provide  a  total  topological  control  structure  trace 
but  merely  print  out  messages  such  as  "Control  can  never 
reach  the  next  statement"  or  "There  is  no  path  to  this  state- 
ment" (See  Figure  6  and  7). 

3.2  Link/ Load  Time  Checks 

In  most  batch-oriented  installations,  compilation  produces  a  binary 
program  file,  usually  stored  on  direct  access  storage.   Subprograms  and 
the  main  program  are  compiled  separately  and  bound  together  by  the  link/ 
loader  which  associates  external  references  and  adjusts  addresses.  If 
an  external  reference  is  to  a  system  library  routine,  it  link/loads  this 
routine.  The  information  available  at  this  point  enables  the  system  to 
perform  the  following  type  of  debugging  or  monitoring  checks: 


3.2.1  Formal  and  Actual  Argument  Checks.  A  subroutine  or  function 
declaration  consists  of  the  name  of  the  subroutine  or  func- 
tion followed  by  a  list  of  actual  parameters. 

For  a  programming  language  such  as  ASA  Standard  FORTRAN, 
the  definition  of  the  language  specifies  that  the  actual 
arguments  must  agree  in  order,  number  and  type  with  the 
corresponding  formal  arguments.  A  check  on  all  three  items 
would  insure  that  the  program  is  adhering  to  a  "standard". 
In  order  to  perform  checks  at  link/load  time,  it  is  necessary 
to  carry  more  information  than  the  usual  external  symbol 
definition,  for  example,  the  list  of  formal  argument  names 
and  their  data  types,  the  corresponding  actual  argument 
names  and  their  data  types.  The  advantages  of  performing  the 
checks  at  link/ load  time  rather  than  execution  time  is  that 
the  check  would  be  done  only  once,  rather  than  each  time 
the  subprogram  is  executed.  The  tests  that  could  be  performed 
include  the  following: 

a)  If  the  subroutine  or  function  does  contain  a  non-standard 
return,  test  that  the  return  label  is  indeed  a  label. 

b)  In  the  case  of  FORTRAN,  if  one  of  the  arguments  of  the 
subroutine  or  function  is  a  subroutine  name  or  a  function 
name,  the  compiler  could  check  to  determine  if  the  name 
is  being  declared  as  an  external  procedure. 

c)  Some  implementations  of  FORTRAN  compilers  allow  the  num- 
ber of  formal  arguments  to  differ  from  the  number  of 
actual  arguments.  If  the  standard  language  definition, 
e.g. ,  ASA  FORTRAN  standard,  calls  for  a  match  in  the  num- 
ber, the  check  might  insure  that  the  program  is  adhering 
to  the  appropriate  standard. 

3.2.2  Static  Subroutine  Structure  Analysis.  The  static  information 
about  subroutine  structure  consists  of  caller  and  call<2e  re- 
lationship derived  from  the  program  text.   In  most  languages, 
this  caller-callee  relationship  is  invariant  at  execution 

time.  This  information  is  available  at  link/ load  time  and 
usually  contained  in  the  external  symbol  definition  table. 
The  link/ loader  could  optionally  produce  a  concordance  of 
the  source  language  subprogram  caller/ callee  names.  An  ex- 
ample of  the  "CALL"  concordance  (Figure  8)  and  "CALLED-BY" 
concordance  (Figure  9)  was  developed  at  NBS  on  the  UNIVAC 
1108  computer. 

3 . 3  Execution  Time  Checks 

_ Dynamic  information  about  the  actual  running  of  the  program  is  ob- 
tainable at  this  time.  To  embed  tests,  the  compiler  usually  inserts 


code  at  appropriate  points  to  be  executed  in  conjunction  with  the 
worker  program.  Every  test  introduce  1  will  decrease  the  running  effi^ 
ciency  of  the  worker  program  and  it  is  necessary  to  exercise  care  when 
deciding  what  to  measure  and  where  to  embed  tests.  Attention  also  has 
to  be  given  to  where  the  tests  are  to  be  inserted  to  assure  uniform 
checks . 

3.3.1  Dynamic  Trace  of  Subroutine  Calls.  Actual  subroutine  paths 
could  be  traced  at  execution  time.  This  information  not  only 
aids  in  debugging  but  is  very  useful  for  program  activity 
analysis.  LEAP  (Lambda  Efficiency  Analysis  Program)  [11], 

a  software  monitor,  contains  a  special  section  to  perform 
subroutine  structure  analysis.  A  tree- like  diagram  of  the 
paths  through  the  hierarchy  of  subprogram  calls  from  the  main 
program  allows  the  user  to  identify  the  most  significant 
call  chains.   Figure  10  shows  a  portion  of  a  dynamic  sub- 
routine trace  developed  at  NBS  for  the  UNI VAC  1108.  It  con- 
sists of  ten  subroutines  with  its  call  chain  properly  reflect- 
ed by  the  indented  printout. 

3.3.2  Backward  Trace  of  Subroutine  Calls  Upon  Error  Termination. 
Upon  error  termination,  an  identification  of  where  the  error 
occurred  plus  a  backward  trace  of  the  subroutine  calls  aids 
debugging.   Figure  11  shows  an  example  of  backward  trace  of 
subroutine  calls  when  the  program  terminated  with  an  error. 
This  example  is  generated  on  the  UNIVAC  1108  FORTRAN  compiler. 
This  compiler  uses  an  extra  word  in  the  calling  sequence  to 
store  the  "walk-back"  or  the  backward  reference  to  permit 
this  kind  of  backward  trace. 

3.3.3  Variable  Trace.  This  is  a  dynamic  display  of  the  specified 
variable  and  its  content  at  each  instant  in  time.  The  display 
usually  occurs  as  an  instruction  by  instruction  accounting  of 
information  or  at  every  instance  of  a  value  change.  It  is 
not  only  useful  for  debugging,  but  also  useful  in  spotting 
the  value  changes  of  certain  variables  for  program  analysis. 

3.3.4  Snapshot.  This  is  similar  to  the  variable  trace  except  that 
the  variables  and  their  values  are  recorded  periodically  on 
entering  or  exiting  certain  regions  of  the  program. 

3.3.5  Flow  Trace.  This  is  the  dynamic  display  of  every  branchpoint 
of  a  running  program.  The  trace  records  the  decision  points 
and  exhibits  the  branches  taken. 

3.3.6  Array  Bounds  Checking.  This  checking  is  sometimes  built  into 
a  computer  as  a  precaution  against  altering  of  values  incor- 
rectly. The  value  of  the  subscript  is  tested  to  determine  if 
it  is  within  the  specified  dimension  of  the  array  element, 
and  if  it  is  also  an  integer  constant.  The  check  is  useful  in 
monitoring  array  elements  and  spotting  the  activity  level  of 
various  parts  of  the  table. 
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3.3.7  Effective  Address  Check.  This  is  a  feature  provided  for  mem- 
ory protection.   Bounds  registers  are  set  to  certain  permitted 
address  ranges  and  every  effective  address  value  is  checked 
against  these  bound  registers.   In  some  cases,  it  is  imple- 
mented as  a  hardware  feature  to  avoid  system  overhead.  This 
feature  is  especially  important  in  a  multiprogramming  environ- 
ment. 

3.3.8  Value  of  the  Control  Variable  Upon  Exit.   In  an  iteration  loop 
such  as  FORTRAN-DO  or  ALGOL-FOR,  the  value  of  the  control 
variable  upon  exit  is  undefined  if  the  exit  is  due  to  exhaus- 
tion of  the  loop,  otherwise  it  is  the  same  as  it  was  immed- 
iately preceding  the  execution  of  the  exit  condition.  The 
undefined  situation  is  handled  by  implementors  in  different 
ways.   Some  compilers  even  try  to  "guess"  what  the  user  in- 
tended. The  trouble  starts  when  programmers  make  use  of  these 
undefined  situations  on  a  particular  implementation  of  the 
compiler,  and  later  discover  that  the  same  program  does  not 
work  correctly  on  a  different  implementation.  Debugging  this 
type  of  error  is  very  difficult  because  it  involves  an  under- 
standing of  the  semantics  of  different  compiler  implementation 
of  undefined  situations.  One  of  the  debugging  facilities  for 
this  particular  undefined  situation  could  be  to  report  all 
later  use  of  the  control  variable  or  intentionally  set  the 
value  of  the  control  variable  to  "undefined"  (minus  zero  or 
some  such  number)  when  the  exit  of  the  loop  is  due  to  exhaus- 
tion of  the  loop. 

3.3.9  GOTO  Checks.  Recently,  there  has  been  considerable  interest 
ui  eliminating  the  GOTO  statement  [12,  13].  Dijkstra  claims 
the  use  of  GOTO  statements  is  undesirable,  and  avoiding  it 
would  increase  the  readability  and  proveability  of  the  program. 
When  GOTO  cannot  be  avoided,  and  when  label  variables  are 
allowed  e.g.  FL/1,  the  following  kinds  of  checks  could  be  per- 
formed by  the  compiler: 

(a)  Flag  error  if  the  transfer  is  made  to  itself  or  to  a  non- 
executable statement. 

(b)  Flag  error  if  the  transfer  label  value  is  negative  or 
undefined. 

(c)  Flag  error  if  the  transfer  label  value  is  outside  of  the 
user's  assigned  program  space. 

(d)  Flag  warning  if  the  transfer  label  goes  within  an  itera- 
tion loop. 

3.3.10  Truncation  Error  Warning.  On  an  arithmetic  or  MOVE-data 
operation,  some  bits  may  be  dropped  due  to  computer  word- 
length.  Overflow  to  the  left  or  to  the  right  of  the  computer 
word  is  called  truncation  error.  If  such  overflow  is  sus- 


pected  or  detected,  a  warning  would  be  printed  by  the  compiler. 

4.   CONCLUSIONS 

Increasing  concern  with  the  quality  of  computer  software  today  makes 
it  important  to  evaluate  critically  the  debugging  facilities  available 
in  high-level  languages.  The  debugging  and  monitoring  aids  described 
above  are  particularly  useful  during  early  implementation  and  initial 
system  integration  stages.  They  could  be  automated  by  embedding  these 
checks  at  appropriate  points  in  the  compilation-execution  process  when 
all  the  needed  information  is  available. 

The  list  of  features  is  not  geared  to  any  particular  high-level  pro- 
gramming language;  however,  some  of  the  features  described  are  applicable 
only  to  particular  language  constructs.  Techniques  of  implementation 
and  the  question  of  how  to  invoke  and  suppress  these  debugging  and  moni- 
toring facilities  have  not  been  addressed  here. 

Commercially  available  compilers  usually  provide  some  debugging  facil- 
ities; however,  certain  trade-off  decisions  are  made  which  usually  sacri- 
fice the  extent  of  providing  debugging  aids  in  favor  of  efficiency  of 
the  compiler.   Such  a  list  might  prove  to  be  useful  as  an  evaluation 
criteria  in  determining  the  capabilities  of  a  compiler. 

These  debugging  and  monitoring  facilities  described  above  could  be 
automated  to  aid  programmers  during  program  development  stage.   Certain 
bugs  are  very  much  problem  and  situation  dependent.   Such  bugs  are  dif- 
ficult even  to  anticipate.   For  instance,  it  would  be  in  general  im- 
possible for  the  compiler  to  isolate  a  sequence  of  code  and  determine 
that  it  is  a  non- terminating  loop.  There  also  exists  a  whole  class  of 
"Timing  Bugs"  which,  when  certain  combination  of  situation  occurs,  the 
program  behaves  abnormally. 

The  problems  in  the  area  of  debugging  and  monitoring  are  many:  The 
question  of  how  much  to  output,  what  should  be  outputted,  how  to  recover 
from  error  conditions,  what  should  be  the  default  situation,  etc.  All 
of  these  questions  are  still  open  and  the  decisions  are  usually  left 
with  the  implementor. 

There  are  other  automated  testing  techniques  for  validating  purposes, 
but  they  are  beyond  the  scope  of  this  paper.   Such  techniques  include 
benchmark  tests,  exhaustive  exercising  of  the  program  with  different  in- 
put data,  and  proof  of  correctness  using  formal  logic,  etc.  Testing 
methods  for  validation  purposes  are  very  different  from  those  mentioned 
here,  which  are  limited  to  debugging  and  monitoring  aids  for  program 
development  purposes. 
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